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Abstract 

This  simulation  experiment  is  the  latest  in  a  series  conducted  by  the  Adaptive  Architecture  for 
Command  and  Control  (A2C2)  research  team.  The  focus  was  to  evaluate  the  relative 
performance  to  two  organizational  structures  on  tasks  that  varied  in  terms  of  complexity  and 
predictability.  One  structure  represented  a  more  traditional,  functional  form  with  interdependent 
nodes.  The  second  structure  was  derived  from  computer- modeling  to  reduce  the  need  for 
coordination  by  creating  fairly  autonomous  divisional  units.  Results  from  a  previous  A2C2 
experiment  suggested  that  the  more  autonomous,  divisional  structure,  while  outperforming  the 
functional  structure  in  planned  mission  tasks,  could  be  less  effective  with  complex  unpredictable 
tasks.  Organization  theory  argues  that  coordination  capability  is  an  important  factor  in  an 
organization’s  ability  to  respond  to  an  uncertain  and  complex  environment.  The  question 
examined  in  this  research  was  whether  the  different  degrees  of  coordination  capability  developed 
by  these  two  structures  would  influence  the  performance  and  process  outcomes  for  both 
predictable  and  unpredictable  tasks.  The  results  show  only  limited  differences  in  the  results  for 
the  two  structures,  though  these  are  in  the  direction  predicted  above.  However,  a  more  consistent 
finding  suggests  that  training  and  improved  teamwork  processes  override  structural  differences 
in  influencing  performance  outcomes.  Implications  to  future  research  and  training  implications 
are  discussed. 

1.  Background 

The  Adaptive  Architectures  for  Command  and  Control  (A2C2)  research  program  has  been 
pursuing  a  stream  of  research  over  the  past  several  years  to  identify  characteristics  of  structure 
and  process  that  will  improve  the  effectiveness  of  defense  organizations.  The  A2C2  team  is 
comprised  of  university,  government,  industry  and  military  participants  doing  basic-level 
research  involving  modeling  and  laboratory  simulation  experiments  as  well  as  applied  research 
as  part  of  operational  activities  such  as  the  Global  Wargame  and  Fleet  Battle  Experiments.  The 
focus  of  this  paper  is  on  the  results  of  the  most  recent  laboratory-based  simulation  experiment 
conducted  at  the  Naval  Postgraduate  School  in  February  2000.  It  builds  significantly  on  NPS- 
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Experiment  4  that  was  conducted  in  1998  and  reported  in  the  1999  CCRT  Symposium  (e.g., 
Entin,  1999;  Hocevar,  Kemple,  Kleinman  &  Porter,  1999).  The  premise  of  this  research  is  that 
defense  operating  environments  will  continue  to  become  increasing  turbulent  and  unpredictable. 
According  to  Vaill’s  (1989)  metaphor,  these  environments  have  the  challenge  of  “permanent 
white  water”  for  the  river  rafter.  Such  environmental  uncertainty  requires  increased 
organizational  adaptability  and  flexibility.  The  question  is,  what  are  the  characteristics  of 
organization  structure  and  process  that  facilitate  adaptability? 

According  to  organizational  theorists  (e.g.,  Galbraith,  1995;  Duncan,  1979,  Mintzberg,  1993), 
organizational  adaptability  can  be  enhanced  through  structural  coordination  mechanisms 
(Galbraith,  1995)  and  processes  of  mutual  adjustment  (Mintzberg,  1993).  Duncan’s  (1979) 
model  argues  that  if  the  task,  market,  or  mission  environment  can  be  segmented,  organizational 
divisions  can  be  structured  with  all  the  necessary  functions  to  meet  the  requirements  of  that 
segment.  A  “divisional”  structure  is  characterized  by  autonomous  units  that  can  readily  respond 
to  changes  in  their  target  environment  when  the  new  demands  or  opportunities  are  within  the 
division’s  resource  capability.  However,  when  the  environment  becomes  increasingly 
unpredictable,  and  the  demands  require  a  response  that  cuts  across  divisional  capabilities,  inter- 
divisional  coordination  capabilities  are  required.  Because  divisional  autonomy  is  emphasized  in 
this  structural  form,  these  coordination  mechanisms  are  not  inherent,  and  thus  this  form  may  not 
perform  effectively  in  the  face  of  task  demands  that  require  inter-divisional  response.  In  fact,  a 
common  criticism  of  the  limitation  of  divisional  structures  is  the  over-emphasis  on  unit-level 
goals  to  the  detriment  of  overall  organizational  performance  (Mintzberg,  1993).  The  traditional 
counter-point  structure  to  a  divisional  form  has  been  the  functional  structure  (Duncan,  1979). 
This  structural  form  inherently  requires  more  coordination  capabilities  when  facing  complex 
tasks  because  organizational  units  are  designed  around  functional  domains  that  are 
interdependent  in  accomplishing  those  tasks.  Historically,  however,  this  type  of  structure  has  not 
proven  to  be  very  “adaptable”  in  responding  to  changing  environments  because  much  of  the 
coordination  was  managed  vertically  through  centralization  of  authority. 

Organizational  theorists  argue  that  whether  a  structure  is  divisional  or  functional,  coordination 
mechanisms  are  needed  for  organizations  to  be  able  to  effectively  respond  to  environmental 
unpredictability  and  complex  task  requirements  that  have  significant  elements  of  uncertainty 
(e.g.,  Burton  &  Obel,  1998).  Lateral  coordination  mechanisms  that  don’t  require  significant 
hierarchic  control,  facilitate  adaptive  response.  These  mechanisms  can  include  as  information 
technology,  linkage  roles,  inter-unit  teams,  and  decentralized  authority  allowing  lower-level 
“mutual  adjustment”  (Galbraith,  1995;  Mintzberg,  1993). 

A2C2  Experiment  4,  conducted  in  1998,  was  designed  to  evaluate  the  effectiveness  of  a  model- 
based  structure  (see  Levchuk  et  al.,  1998)  in  the  performance  of  a  simulated  Joint  Task  Force 
(JTF)  mission.  This  structure  (Al-6)  was  derived  from  computer  modeling  that  analyzed  mission 
task  requirements  and  the  resource  options  and  used  design  criteria  of  workload  balance,  speed 
of  execution,  and  minimized  coordination  requirements.  Following  organization  theory 
concepts,  the  Al-6  structure  can  be  defined  as  having  a  divisional  form  with  the  divisions  being 
defined  based  on  tasks.  In  other  words,  nodes  (i.e.,  divisions)  within  the  command  structure 
were  assigned  assets  such  that  they  were  able  to  accomplish  fairly  complex  tasks  without 
significant  coordination  with  other  command  cells.  The  hypothesis  of  Experiment  4  was  that  this 


model-based  structure  (Al-6)  would  outperform  an  alternative  structure  (AO-6)  that  was  more 
functionally  defined  and  required  significantly  more  coordination  to  accomplish  the  major 
planned  mission  tasks.  The  justification  for  this  hypothesis  was  that  Al-6  would  operate  more 
efficiently  with  unit-level  autonomy  in  accomplishing  the  planned  mission  tasks.  This 
hypothesis  was  supported  by  the  results  of  Experiment  4  (see  Entin,  1999;  Hocevar  et  al.,  1999). 
These  results  reinforce  the  design  benefits  of  a  divisional  structure  when  the  environment  or  task 
set  can  be  segmented  and  the  environment  is  fairly  predictable.  This  finding  has  important 
implications  to  the  design  of  Joint  Task  Forces  and  supports  the  concept  of  mission-defined 
units. 

However,  interesting  tentative  findings  based  on  a  very  limited  set  of  simulation  events  also 
emerged.  When  faced  with  unpredicted  tasks  that  required  inter-nodal  coordination,  the 
performance  of  the  more  autonomous  task-based  structure  (Al-6)  fell  below  that  of  the  AO-6 
structure.  While  this  finding  was  not  statistically  significant  due  to  limits  in  sample  size  and 
statistical  power,  it  is  congruent  with  theories  of  organization  design  that  propose  coordination 
capabilities  provide  an  adaptive  mechanism  for  responding  to  environmental  and  task  uncertainty 
(e.g.,  Galbraith,  1977,  1995;  Duncan,  1979;  Burton  &  Obel,  1998).  In  other  words,  theory 
would  argue  that  the  higher  degree  of  interdependence  among  the  organizational  units  in  the 
more  functionally  defined  AO-6  allowed  that  JTF  structure  more  practice  in  coordination  as  they 
accomplished  the  planned  mission  tasks.  This  greater  practice  in  coordination  could  have 
enhanced  this  structure’s  facility  in  responding  to  the  complex  unpredicted  events  requiring 
inter-nodal  coordination  that  emerged  in  the  scenario. 

This  explanation  has  been  reinforced  by  the  recent  work  of  Ensign  (1998)  who  argues  that 
organizations  need  to  develop  not  only  the  capabilities  to  manage  existing  interdependence,  but 
potential  interdependence.  Similarly,  Weick’s  and  Roberts’s  (1993)  work  on  aircraft  carriers 
suggests  that  situations  that  are  fluid,  high  risk,  and  highly  interdependent  require  teams  to  be 
“heedful”  of  threatening  events  and  practiced  in  creating  a  joint  response  to  these  events.  The 
question  raised  by  the  tentative  findings  in  Experiment  4  was  whether  the  model-based  structure 
(Al-6)  with  high  unit- level  autonomy  for  planned  events  had  overly  constrained  the  practice  of 
coordination  and  thus  diminished  the  organization’s  ability  to  respond  to  unpredicted  events. 

2.  Study  Purpose  and  Design 

The  purpose  of  this  experiment  was  to  gather  more  data  on  the  relationship  between  structural 
types  and  performance  on  unpredicted  tasks.  Specifically,  the  research  question  guiding  the 
design  of  the  experiment  was:  When  faced  with  the  need  to  respond  to  an  unanticipated, 
complex,  task,  does  a  structural  architecture  that  requires  greater  levels  of  inter-unit 
coordination  (i.e.,  nodes  with  structural  interdependence)  provide  a  performance 
advantage  over  an  architecture  that  reduces  coordination  by  using  a  task-based  design  (i.e., 
nodes  with  structural  autonomy)? 

Study  participants  were  36  military  officers  (02-05)  in  varying  Master  of  Science  programs  at 
the  Naval  Postgraduate  School.  These  officers  represent  USN,  USA,  and  USMC  as  well  as 
foreign  national  officers  from  several  nations  including  Germany,  Brazil,  Canada,  and  Greece. 
Students  were  randomly  assigned  to  6-person  teams  maintaining  a  balance  across  teams  in  terms 


of  service  represented,  national  origin,  rank,  and  support  vs.  operational  experience.  Each  team 
played  the  role  of  a  Joint  Task  Force  engaged  in  a  mission  using  the  Distributed  Dynamic 
Decisionmaking-III  (DDD-III)  simulation  software  (Kleinman,  Young  &  Higgins,  1996). 

2.1  Independent  variables:  Organizational  structure 

The  two  command  structures  used  in  this  experiment  are  presented  in  Figure  1.  These  structures 
were  identical  to  those  used  in  the  previous  Experiment  4  described  above.  Specifically,  AO-6 
represents  a  more  functionally-defined  structure  where  the  nodes  are  interdependent  in  the 
accomplishment  of  complex  mission  tasks  while  A 1-6  is  more  divisional  with  units  designed  for 
more  autonomous  performance  of  planned  mission  tasks.  There  were  two  structural  changes 
made  for  this  experiment.  First,  in  Experiment  4,  both  command  structures  used  two  separate 
communication  nets  that  followed  the  command  structure.  Flag  and  the  immediate  subordinate 
nodes  were  on  one  net  and  remaining  nodes  were  on  separate  net  with  command  node  ‘Blue’ 
acting  as  the  communication  link-pin.  In  the  current  experiment,  a  single  communication  net 
was  used  for  both  structures.  The  second  change  from  Experiment  4  was  the  shift  of  the  DDG 
from  Blue  to  Green  in  AO-6.  This  was  done  in  response  to  participant  feedback  in  Experiment  4 
on  the  heavy  workload  on  Blue.  No  changes  were  made  to  the  model-based  architecture  A 1-6 
(see  Levchuk  et  al,  1998  for  details  on  the  modeling  procedure). 


Figure  1.  Organizational  Architectures 


AO-6  Functionally  Interdependent  Al-6  Task-based  Autonomous 

(model-based) 


The  role  of  hierarchic  or  centralized  command  and  control  was  not  a  manipulated  parameter  in 
this  experiment.  To  try  to  minimize  the  variation  due  to  leadership  style,  the  following 
instructions  were  read  to  all  participants  prior  to  the  first  mission  rehearsal  session: 

It  should  be  presumed  that  the  JTF  has  pre-planned  this  mission,  and  with  clear 
commander’s  intent,  authority  has  been  delegated  to  subordinate  nodes  to  laterally 
coordinate  required  assets  on  planned  mission  tasks.  This  lateral  coordination,  or 


self-synchronization,  can  be  accomplished  without  seeking  approval  of  higher 
authority.  HOWEVER,  this  does  not  remove  Flag’s  (or  Blue’s)  responsibility  to 
intervene  when  he/she  feels  the  mission  requires  refocusing  or  to  resolve  conflicts 
and  set  priorities  when  assets  are  required  for  multiple  tasks. 

The  experimental  design  used  “within-teams”  comparison;  all  teams  executed  the  simulation 
using  both  structures  —  one  designed  for  autonomy  (Al-6),  and  one  designed  for 
interdependence  (AO-6)  -  with  order  balanced  across  teams  to  control  for  any  learning  effect. 

2.2  Mission  Scenario 

The  mission  of  the  JTF  was  a  scenario  in  which  the  U.S.  is  taking  action  in  support  of  an  ally, 
Country  Green,  that  has  been  invaded  by  neighboring  Country  Orange.  The  ultimate  objectives 
of  this  mission  are  to  secure  Country  Green’s  Airport  and  Port.  A  mission  briefing  document 
that  outlined  a  specific  chronology  of  mission  tasks  to  be  undertaken  by  the  JTF  was  distributed 
to  all  participants.  A  simplified  version  is  listed  below: 

1.  Amphibious  forces  will  land  and  take  North  and  South  Beach  after  clearing 
mines. 

2.  Prior  to  taking  N.  Beach,  infantry  (INF)  and  air  support  will  seize  the  hill 
overlooking  the  beach. 

3.  Infantry  will  move  down  roads  from  S.  Beach  toward  Airport  and  from  N. 

Beach  to  Port  clearing  mines  and  enemy  tanks. 

4.  Special  Operations  Force  (SOF)  and  satellite  (SAT)  must  determine  which  of 
two  roads  the  enemy  plans  to  use  for  insertion  of  forces  by  assessing  traffic. 

Once  the  enemy  “lead  vehicle”  is  identified,  it  should  be  destroyed  as  well  as 
the  bridge  being  used  by  that  vehicle,  while  retaining  second  bridge  for 
friendly  traffic. 

5.  Armored  counterattack  forces  are  believed  to  be  at  the  Airport  and  Port.  If 
present,  the  must  be  identified  and  destroyed. 

6.  Both  the  Port  and  Airport  must  be  captured  and  held.  The  attack  on  the 
Airport  has  priority  and  should  occur  first  if  they  cannot  be  attacked 
simultaneously. 

The  above  mission  tasks  are  assumed  to  represent  a  high  level  of  certainty.  They  all  MUST 
occur,  and  the  sequencing,  location,  and  assets  required  for  each  task  are  clearly  specified  and 
documented  for  team  participants.  With  the  exception  of  clearing  the  land  and  sea  mines,  all  of 
the  above  mission  tasks  are  somewhat  complex  in  that  they  required  multiple  assets  to 
accomplish  (e.g.,  infantry  (INF)  and  close  air  support  (CAS)). 

The  mission  briefing  document  also  defined  “Other  Tasks  That  Can  Occur  Throughout  The 
Operation.”  These  “defensive”  tasks  include  such  tasks  as  the  identification  and  prosecution  of 
hostile  aircraft  and  patrol  boats  and  the  elimination  of  enemy  artillery  sites.  These  tasks  have 
some  limited  unpredictability  in  that  the  location  and  timing  is  not  known,  but  the  likelihood  of 
their  appearance  is  certain.  The  majority  of  these  tasks  are  “simple”  in  that  they  require  only  the 
application  of  one  asset  and  can  thus  be  accomplished  by  a  single  organizational  command  node. 


However,  there  are  three  defensive  tasks  (enemy  tanks.  Silkworm  and  SAM  missile  sites)  that 
are  also  somewhat  unpredictable,  but  require  the  use  of  multiple  assets. 


The  mission  scenario  used  in  Experiment  4  was  embellished  with  four  additional  complex, 
unexpected,  and  unrehearsed  tasks.  These  tasks  were  not  introduced  until  after  each  team  had 
completed  2  hours  of  basic  software  training  and  had  conducted  two  mission  rehearsals  using  the 
first  structure  they  had  been  assigned.  Just  prior  to  having  the  team  conduct  the  mission  for  the 
actual  data  gathering,  an  announcement  was  made2: 

As  in  any  real  mission,  we  cannot  assume  that  what  we  rehearse  perfectly  predicts 
the  events  that  will  unfold.  You  should  expect  unanticipated  task  requirements  as 
you  conduct  this  mission.  The  handout  [given  each  participant]  shows  the  icon 
that  indicates  an  “Unanticipated  Task”  (UT)  and  explains  how  to  use  the  software 
to  gather  intelligence  on  both  the  nature  of  the  task  and  the  resources  required  to 
prosecute  the  task.  This  task  is  equally  important  to  all  your  mission  tasks,  but 
does  not  change  the  overall,  planned  mission  requirements.  UTs  are  time-critical. 

Each  task  bearing  this  new  icon  will  expire  seven  minutes  after  it  appears. 

After  this  announcement,  each  team  had  a  few  minutes  to  discuss  a  general  strategy  for 
addressing  UTs.  The  total  time  for  each  team  to  accomplish  the  mission  scenario  was  between 
33  and  40  minutes  (mean  37  minutes). 

2.3  Complex  Unanticipated  Tasks 

A  key  question  to  be  addressed  in  this  experiment  is  to  compare  the  performance  of  the  two 
structures  on  tasks  that  are  both  complex  and  unanticipated  and  unrehearsed.  These  tasks  (UTs) 
were  designed  into  the  DDD-III  simulation  software  and  had  the  following  characteristics: 

•  UTs  appeared  with  a  distinctive  icon  illustrated  on  the  handout  given  to  participants. 

•  There  were  no  additional  cues  other  than  the  visual  icon. 

•  UTs  first  had  to  be  scanned  by  the  Satellite  asset  before  “intelligence”  was  available. 

•  Once  scanned,  a  brief  description  of  the  task  and  the  possible  asset  packages  to  be  used 
were  available  using  the  help  window  for  the  task. 

•  All  UTs  had  to  be  prosecuted  within  7.5  minutes  (450  second)  or  the  task  score  was  zero. 

•  Each  UT  required  the  coordination  of  a  minimum  of  2-3  nodes. 

•  UTs  were  constructed  to  test  the  teams’  ability  to  coordinate  in  ways  previously 
unrehearsed  (i.e.,  with  asset  combinations  not  used  in  the  accomplishment  of  the  primary 
mission  tasks  that  were  the  focus  of  previous  training). 

•  The  4  UTs  in  each  scenario  were  distinctive  from  each  other 

•  A  unique  set  of  4  UTs  was  used  for  each  team’s  trial  run  with  the  second  organizational 
structure;  the  two  sets  of  4  UTs  were  balanced  in  terms  of  difficulty  (as  measured  by  the 
amount  of  coordination  required). 


2  It  is  recognized  that  this  announcement  diminished  the  “unpredictability'’  of  these  events.  However,  it  was 
necessary  to  inform  participants  of  the  meaning  of  the  screen  icon  that  represented  “Unanticipated  Task”  (UT)  and 
give  them  the  procedural  guidelines  for  gathering  intelligence  using  the  software. 


Examples  of  two  of  the  UTs  are  given  below: 


UT1  [information  from  Task  Help  screen]:  Fishing  trawler  laying  mines  in 
shipping  lane.  Establish  air  superiority  to  protect  the  SMC  [mine  sweeper]. 
Destroy  enemy  trawler  and  clear  the  mines.  Time  is  critical  -  oil  tankers  are 
entering  the  area. 

SUITABLE  ASSET  PACKAGES  FOR  THIS  TASK: 

{ SMC+CG+DDG }  OR  { SMC+2VF+DDG }  OR  [SMC+2VF+CG] 

UT2:  F14  crew  is  downed  in  hostile  waters  with  casualties  reported.  Protect 
SAR  [search  and  rescue]  operation  from  enemy  air  and  surface  actions  -  time  is 
critical! 

SUITABLE  ASSET  PACKAGE  FOR  THIS  TASK: 

[MED+VF+CAS] 

2.4  Dependent  Measures 

Three  categories  of  measures  will  be  examined  in  this  report  -  those  derived  from  the  simulation 
software,  observer  ratings  and  participant  ratings.  First,  the  DDD-III  software  generates  two 
overall  performance  scores  -  Mission  and  Strength.  The  Mission  score  generally  reflects 
offensive  performance  based  on  the  extent  to  which  all  tasks  are  accomplished  with  appropriate 
resources  and  timely  coordination.  At  the  beginning  of  a  simulation.  Mission  score  starts  at  zero 
and  cumulates  to  a  possible  score  of  100%.  The  Strength  score  generally  reflects  defensive 
performance;  it  starts  at  100%  and  decrements  accrue  as  a  result  of  enemy  hits,  collisions,  or 
mission  tasks  attempted  with  inadequate  resources  or  coordination.  Neither  the  Mission  nor  the 
Strength  scores  include  performance  of  the  four  Unanticipated  Tasks. 

In  addition  to  these  overall  measures,  specific  task  performance  can  be  derived  from  the  DDD 
dependent  variable  file  generated  by  the  software  as  teams  perform  the  simulation.  Two 
measures  of  interest  will  be  evaluated  here  -  accuracy  and  latency.  Accuracy  scores  are  based  on 
100%  and  are  reduced  if  assets  are  not  appropriate  or  timely,  or  when  collisions  or  enemy 
penetrations  occur  related  to  that  task.  In  addition,  some  task  categories  (e.g.,  patrol  boats) 
represent  multiple  rather  than  single  events.  The  accuracy  scores  for  these  tasks  are  reduced 
proportionately  when  task  events  are  “missed”  (e.g.,  an  enemy  patrol  boat  gets  through  to  the 
resupply  port).  Latencies  reflect  the  time  taken  to  accomplish  a  task. 

To  evaluate  tasks  of  differing  degrees  of  complexity  and  unpredictability,  the  following 
categories  of  tasks  were  evaluated: 

•  7  Primary  Mission  Tasks  (complex,  predictable  and  rehearsed) 

•  6  Single-node  Defensive  Tasks  (simple,  somewhat  unpredictable,  but  rehearsed) 

•  3  Coordinating-nodes  Defensive  Tasks  (moderate  complexity,  somewhat 
unpredictable,  rehearsed) 

•  4  Unanticipated  Tasks  (complex,  unpredictable,  unrehearsed) 


For  each  data  trial,  two  observers  independently  evaluated  both  performance  quality  and 
teamwork  processes  on  7-point  rating  scales  (see  Entin  et  al.,  1998;  Entin  el  al..  1997).  These 
observers  were  military  officers  who  were  well- versed  in  the  mission  requirements  and  software 
operation.  The  performance  instrument  included  10  behaviorally-anchored  scales  assessing 
performance  of  specific  mission  tasks  that  represented  all  of  the  first  three  task  categories  cited 
in  the  bullet-list  above.  The  teamwork  instrument  included  seven  behaviorally-anchored  scales 
that  assessed  several  dimensions  of  process  behaviors  including:  back-up  behaviors, 
communication  effectiveness,  coordination,  and  team  orientation. 

Finally,  participants  were  asked  to  complete  assessment  forms  at  the  completion  of  the  data  trial 
for  each  structure  to  evaluate  both  situation  awareness  and  workload.  At  the  conclusion  of  the 
second  (final)  data  trial,  they  also  completed  a  post-experiment  questionnaire  that  asked  them  to 
compare  the  two  structures  on  a  number  of  dimensions  related  to  both  performance  and  process. 
This  form  used  a  7-point  rating  scale  where  1=  “low”  and  7=  “high”  on  the  focal  characteristic. 

3.  Results 

3.1  Difference  in  Coordination  Practiced  Using  Two  Structures 

The  first  test  conducted  was  to  determine  the  degree  of  difference  between  the  two  structures  in 
the  amount  of  coordination  “practiced”  as  the  teams  conducted  rehearsal  missions.  This  test  is 
important  in  order  to  demonstrate  if  the  key  factor  of  “coordination  capability”  could  be  argued 
as  distinctive  for  the  two  structures.  Only  given  this,  would  it  be  possible  to  test  if  the  difference 
in  coordination  practice  would  relate  to  performance  on  both  the  predictable  primary  mission 
tasks  and  the  Unanticipated  Tasks.  The  average  number  of  command  nodes  used  in  the  conduct 
of  each  mission  scenario  was  calculated  from  the  DDD  dependent  variable  file  in  three  ways  - 
for  the  seven  primary  mission  tasks,  for  all  task  categories,  and  for  all  task  events.  The  third 
calculation  differs  from  the  second  in  that  many  of  the  non-primary  tasks  have  multiple 
occurrences.  In  the  second  calculation  a  task  category  (e.g.,  enemy  patrol  boats)  is  counted  only 
once.  In  the  third  calculation,  the  average  reflects  the  number  of  actual  patrol  boat  tasks 
prosecuted.  The  three  comparisons  of  the  amount  of  coordination  used  by  the  two  structures  is 
presented  in  Table  1. 


Table  1.  Mean  Number  of  Nodes  Coordinating  on  Mission  Tasks  by  Structural  Type 

AO-6 

(interdependent) 

Al-6 

(autonomous) 

t-test 

sig 

Seven  Primary  Mission  Tasks 

2.28 

1.52 

p<.001 

All  Task  Categories 

1.67 

1.32 

pc. 001 

All  Task  Events 

1.42 

1.30 

pc.003 

The  results  of  this  analysis  show  that  there  is  a  statistically  significant  difference  in  the  amount 
of  coordination  (i.e.,  mean  number  of  nodes  involved  in  prosecuting  tasks)  used  by  the  two 
structures,  for  all  three  calculations.  The  difference  is  most  substantial  for  the  primary  mission 
tasks.  The  result  is  reflective  of  the  structural  design  differences  of  more  interdependence  in  A0- 

3  It  was  not  possible  to  analyze  the  Situation  Awareness  data  in  time  for  inclusion  in  this  paper.  In  addition,  all 
communications  during  the  conduct  of  each  mission  trial  were  coded  independently  by  two  members  of  the  research 
team.  These  process  data  are  not  included  in  the  analysis  presented  here. 


6  (i.e.,  more  coordination  among  nodes)  and  more  autonomy  in  Al-6.  However,  it  is  important 
to  note  that,  for  both  structures,  there  were  a  significant  number  of  “simple”  tasks  requiring  only 
one  node.  Thus,  the  overall  practice  of  coordination  reflected  by  All  Task  Events,  while 
statistically  significant,  shows  only  a  small  real  difference  between  the  two  structures. 

3.2  Simulation  Performance  Comparing  Two  Structures 

The  results  of  performance  derived  from  the  DDD  simulation  software  are  presented  first.  As 
described  above,  Mission  and  Strength  reflect  overall  measures  of  performance  (excluding  the 
Unanticipated  Tasks).  Specific  task  categories  are  analyzed  separately  in  order  to  examine 
whether  the  structures  performed  differently  depending  on  the  complexity  and  predictability  of 
the  tasks.  The  results  of  the  DDD-based  measures  are  presented  in  Table  2. 


Table  2.  Simulation  Performance  Comparing  Two  Structures 

AO-6 

Al-6 

(interdependent) 

(autonomous) 

Mission  score 

89.8 

92.2 

(6.1) 

(5.6) 

Strength  score 

87.0 

89.8 

(6.1) 

(4.1) 

7  Primary  Offensive  Mission  Tasks  (predictable)  -Accuracy 

92.6 

96.7 

(11.7) 

(5.0) 

7  Primary  Offensive  Mission  Tasks  (predictable)  -Latency 

1065 

1048 

(116) 

032) 

4  Complex  Unanticipated  Tasks  (unpredictable)  -Accuracy 

90.8 

86.0 

(11.7) 

(13.6) 

4  Complex  Unanticipated  Tasks  (unpredictable)  -Latency 

283 

303 

(42) 

(45) 

Single-node  Defensive  Tasks  -Accuracy 

87.3 

92.4 

(8.6) 

(6.6) 

Single-node  Defensive  Tasks  -Latency 

195 

179 

(46) 

(16) 

71.6 

82.2 

(9.0) 

(11.7) 

Coordinating-nodes  Defensive  Tasks  -Latency 

368 

391 

(127) 

(179) 

Note:  Means  and  (std  dev.).  Accuracy  based  on  100%;  Latency  is  number  of  seconds 

Paired  (dependent  samples)  t-test  comparison  of  means  showed  all  differences  to  be  non-significant 
(p<.  10),  (2 -tailed  test).  N=6. 

Table  2  presents  means  and  standard  deviations  for  both  performance  accuracy  measures  and 
latencies.  Paired  t-test  comparisons  showed  no  significant  differences  in  the  means  (pc.  10). 
There  are  several  possible  explanations  for  these  results.  First,  because  of  the  small  sample  size 
(N=6),  there  is  limited  statistical  power  to  show  “true”  differences  in  performance.  Second,  as 
shown  by  the  high  accuracy  scores,  and  given  the  feedback  from  participants,  there  could  be  a 
“ceiling  effect.”4  In  other  words,  a  more  challenging  scenario  may  be  necessary  to  distinguish 
the  performance  of  the  structures.  Third,  any  possible  differences  between  the  structures  were 


4  All  teams  received  more  comprehensive  pre-trial  training  than  in  previous  A2C2  experiments.  This  training 
emphasized  both  the  unique  asset  responsibilities  of  each  command  node  as  well  as  practicing  coordination  for 
complex  tasks. 


overridden  by  the  strong  team  cohesion  established  and  coordination  practiced  by  all  teams 
during  training  and  rehearsal. 

Even  given  the  lack  of  statistical  evidence,  it  is  interesting  to  examine  the  pattern  of  means  in 
terms  of  the  research  question  of  interest.  First,  for  both  the  overall  measures  of  Mission  and 
Strength  as  well  as  all  accuracy  measures  except  those  for  the  Unanticipated  Tasks,  the  model- 
based  structure  of  A 1-6  is  slightly,  but  consistently  more  effective  than  the  AO-6  structure.  In 
contrast,  the  comparison  of  means  for  the  four  Unanticipated  Tasks  shows  a  different  trend.  In 
this  case,  AO-6  shows  a  somewhat  higher  accuracy  score  and  a  lower  latency  (i.e.,  greater  speed 
of  task  accomplishment)  than  Al-6.  This  pattern  could  suggest  that  a  structure  with  more 
practice  in  coordination  could  enhance  the  ability  to  respond  to  unanticipated,  but  complex  task 
requirements;  and  that  more  autonomy  facilitates  performance  of  tasks  that  are  both  complex  and 
predictable  as  well  as  simple  tasks  of  limited  unpredictability  tasks.  However,  it  is  not  possible 
to  draw  definitive  conclusions  from  these  findings  given  the  lack  of  statistical  evidence. 

3.3  Observer  Ratings  of  Performance  and  Process  Comparing  Two  Structures 

Observer  performance  rating  scales  developed  by  Entin  (1998)  included  10  questions  using  a  7- 
point  rating  scale.  One  question  pertained  to  overall  mission  performance.5  The  others  focused 
on  a  selection  of  specific  tasks  representing  both  primary  mission  tasks  and  defensive  tasks.  As 
described  in  the  discussion  of  scenario  and  measures,  these  two  task  categories  differ  in  term  of 
the  degree  of  predictability.  Participants  were  aware  of  the  defensive  tasks  and  their  role  in 
addressing  them;  however  they  did  not  know  the  exact  location  and  timing  of  those  tasks. 
Because  observers  were  not  matched  with  specific  teams,  the  comparison  of  means  used  an 
independent  samples  test.  The  results  of  observer  ratings  are  presented  in  Table  3. 


Table  3.  Observer  Ratings  of  Performance  Comparing  Two  Strucl 

tures 

AO-6  (interdependent) 

Al-6  (autonomous) 

T-test 

significance 

Overall  Mission 

5.33 

5.98 

p<.02 

Primary  Mission  Tasks  (predictable) 

6.13 

6.35 

n.s. 

Defensive  Tasks  (less  predictable) 

4.73 

5.82 

pc.Ol 

Note:  Means  based  on  a  rating  scale  where  1=  “poor”  and  7=  “superior.” 

Independent  samples  t-test;  2-tailed  significance.  N=6. 

In  contrast  with  the  results  derived  from  the  simulation  software,  there  were  statistically  different 
results  for  the  comparison  of  the  two  structures  using  observer  data.  Specifically,  the  model- 
based  structure  (Al-6)  with  task-defined  divisional  units  showed  stronger  performance  ratings 
for  both  the  overall  mission  question  and  the  assessments  of  the  defensive  tasks.  These 
statistically  significant  findings  give  greater  credibility  to  the  similar  pattern  of  means  derived 
from  the  DDD  software  presented  in  Table  2,  particularly  for  the  defensive  tasks.  The  observer 
ratings  of  the  performance  of  the  Primary  Mission  Tasks  showed  no  difference  between 
structures.  It  is  not  possible  to  use  the  overall  mission  performance  rating  by  observers  because 
this  included  their  assessment  of  teams’  processing  of  Unanticipated  Tasks  as  well  as  all 


5  The  observers  were  not  asked  to  separately  evaluate  UT  performance,  but  UT  tasks  were  included  in  their  rating  of 
overall  performance. 


rehearsed  mission  and  defensive  tasks.  In  the  DDD-derived  measures,  the  overall  measures  (e.g., 
Strength,  Mission)  do  not  include  Unanticipated  Tasks  performance. 


To  further  our  understanding  of  the  particular  processes  that  contribute  to  performance 
differences,  observers  also  rated  each  team’s  scenario  performance  on  a  number  of  questions 
related  to  teamwork.  The  results  of  these  ratings  compared  by  structure  are  presented  in  Table  4. 


Table  4.  Observer  Ratings  of  Processes  Comparin; 

g  Two  Structures 

AO-6 

(interdependent) 

Al-6 

(autonomous) 

T-test 

significance 

Overall  Teamwork 

5.14 

5.56 

n.s. 

Members  anticipate  others'  need  for  assistance 

4.67 

5.50 

p=.19  (n.s.) 

Members  adjust  to  redistribute  workload 

4.50 

5.58 

to 

II 

© 

SO 

* 

Communication  Effectiveness 

4.42 

5.67 

p=.14  (n.s.) 

Proactive  Communication 

5.42 

5.17 

n.s 

Coordination 

5.58 

5.92 

n.s 

Team  Orientation 

5.92 

6.08 

n.s. 

Willingness  to  shift  from  main  mission  tasks  to 

UTs 

6.02 

5.79 

n.s. 

Extent  to  which  team  considered  alternatives  in 
processing  UTs 

4.67 

5.10 

n.s. 

Teams  level  of  activity  when  UTs  appeared 

5.02 

4.77 

n.s. 

Note:  Means  based  on  a  rating  scale  where  1=  “poor”  and  7=  “superior.” 

Independent  samples  t-test;  2-tailed  significance  (p<.10).  N=6. 

Only  one  measure  shows  a  difference  that  achieves  statistical  significance  (pc.  10).  Specifically, 
observers  rated  teams  using  the  model-based  structure  (Al-6)  as  significantly  stronger  in  their 
awareness  of  others’  workload  and  willingness  to  adjust  to  keep  workload  balanced.  The  mean 
ratings  show  a  similar  pattern  for  the  related  “back-up”  behavior  of  anticipating  others’  need  for 
assistance,  but  the  t-test  is  just  below  the  critical  value  for  significance.  Two  other  teamwork 
behaviors  that  support  the  advantage  of  the  Al-6  structure  are  general  communication 
effectiveness  and  coordination,  though  again  not  achieving  statistical  significance.  However, 
given  the  limited  sample  size,  this  consistent  pattern  of  effective  teamwork  is  noteworthy.  This 
result  suggests,  given  the  principles  of  organization  theory,  that  the  more  limited  coordination 
practiced  by  the  divisional  structure  designed  to  increase  unit-level  autonomy,  was  sufficient  to 
develop  participants’  commitment  to  team  rather  than  unit-level  goals.  The  additional 
coordination  practice  for  teams  in  AO-6  structure  did  not  seem  to  yield  an  advantage  in  these 
domains  of  teamwork  process.  However,  no  structural  advantage  seems  evident  in  mean 
observer  ratings  of  processes  evident  in  the  prosecution  of  Unanticipated  Tasks. 

3.4  Participant  Self-Report  Data  Comparing  Two  Structures 

At  the  end  of  each  team’s  application  of  the  second  structure  to  the  simulated  mission, 
participants  completed  a  survey  comparing  the  two  structures.  They  were  asked  to  rate  each 
structure  on  a  set  of  questions  where  1=  “low”  on  the  stated  characteristic  and  7  =  “high”  on  the 
stated  characteristic.  Table  5  presents  the  results  of  paired  comparison  t-tests  using  two-tailed 
test  of  significance  (pc.lO).  Only  one  of  the  measures  achieves  a  level  of  statistical  significance. 
Item  number  2  in  Table  5  shows  that  the  AO-6  structure  where  nodes  were  more  interdependent 
and  there  was  greater  practice  of  coordination,  was  rated  as  more  effective  in  performing  the 


Unanticipated  Tasks  (pc.  10).  This  finding  supports  organization  theorists  who  argue  that  the 
ability  to  respond  to  complex  and  uncertain  task  environments  is  enhanced  by  coordination 
mechanisms  (e.g.,  Ensign,  1998;  Galbraith,  1995;  Mintzberg,  1993).  It  also  strengthens  the 
interpretation  of  the  non-significant  software -based  performance  measures  in  Table  2  that 
showed  both  greater  accuracy  and  faster  latency  for  AO-6  for  the  UTs.  Three  other  items  in 
Table  5  show  means  in  the  direction  of  the  hypothesis  (though  not  significant)  related  to  the 
value  of  greater  coordination  capability.  Specifically,  item  5  (facilitation  of  communication), 
item  8  (situational  awareness),  and  item  10  (willingness  to  shift  from  planned  tasks  to  UTs)  have 
higher  participant  ratings  for  AO-6  as  compared  with  A 1-6.  However,  other  items  that  could  also 
be  expected  to  correlate  with  greater  coordination  practice  (e.g.,  flexibility,  coordination)  do  not 


Table  5.  Participant  Ratings  of  Two  Structures 

AO-6 

(interdependent) 

Al-6 

(autonomous) 

1.  How  well  did  each  structure  perform  mission? 

5.49 

5.58 

2.  How  well  did  each  structure  perform  UTs? 

5.78 

5.48* 

3.  Flexibility 

5.07 

5.05 

4.  Structure's  facilitation  of  coordination 

5.29 

5.34 

5.  Structure's  facilitation  of  communication 

5.40 

5.23 

6.  Structure's  facilitation  of  development/use  of  efficient  strategies 

5.35 

5.39 

7.  Balance  of  task  distribution  across  structural  nodes 

5.03 

4.88 

8.  Situational  awareness 

5.42 

5.25 

9.  Extent  to  which  UTs  interfered  with  execution  of  planned  mission 
tasks 

4.48 

4.40 

10.  Willingness  to  shift  from  planned  tasks  to  UTs 

5.88 

5.71 

11.  To  what  extent  did  team  consider  alternatives  in  processing  UTs? 

4.38 

4.54 

12.  Degree  of  Flag  involvement  in  processing  UTs? 

5.32 

5.38 

Note:  Means  are  based  on  a  rating  scale  where  1=  “low”  and  7  =  “high”  value  on  the  characteristic. 

*  Paired  (dependent  samples)  t-test  significant  g<.10  (2-tailed).  All  other  t-tests  not  significant. 

N’s  ranged  from  29  to  36  given  missing  cases. 

show  a  similar  pattern.  It  is  also  interesting  to  note  that  data  from  the  participants  do  not  confirm 
the  observer  ratings  of  more  effective  performance  of  the  primary  mission  tasks  under  the  A 1-6 
model-based,  more  autonomous  structure.  The  means  on  this  item  (#1)  shows  no  significant 
difference  between  the  two  structures. 

3.5  Assessing  the  Effect  of  Training  on  Performance 

One  obvious  design  limitation  to  this  experiment  is  that  once  participants  have  experienced 
Unanticipated  Tasks,  their  degree  of  uncertainty  is  greatly  diminished.  In  addition,  the  practice 
that  teams  get  in  the  coordination  required  to  accomplish  these  tasks  in  whatever  structure  they 
are  assigned  in  Trial  1  will  likely  improve  their  ability  to  perform  comparable  tasks  in  Trial  2. 
Steps  were  taken  in  the  design  of  the  UTs  such  that  the  “cover  story”  for  each  task  from  Trial  1 
to  Trial  2  was  different,  as  were  the  timing  and  location  within  the  scenario.  Nevertheless,  the 
possibility  of  a  training  effect  diminishing  the  influence  of  structure  on  UTs  needed  to  be 
examined.  Table  6  presents  results  for  the  same  measures  derived  from  the  DDD-software  that 
were  presented  in  Table  2.  However,  in  this  case  the  comparison  combines  both  structures  used 
in  Trial  1  and  compared  them  with  both  structures  used  in  Trial  2.  Because  of  the  directional 
nature  of  this  test,  t-test  comparisons  use  a  one-tailed  significance  test  for  pc.  10. 


The  results  of  the  paired  t-test  comparisons  in  Table  6  show  that  there  were  some  significant 
learning  effects  evident  for  specific  measures  of  performance.  The  most  consistent  finding  was 
that  three  of  the  four  latency  measures  show  a  significant  improvement  (i.e.,  decrease)  in  Trial  2 
and  the  fourth  is  in  the  same  direction,  though  not  statistically  significant  (pc.  10).  Of  the  six 
performance  measures  (mission,  strength,  and  4  accuracy  scores),  only  the  single-node  defensive 
task  measure  shows  a  statistically  significant  improvement,  though  the  improvement  in  the 
accuracy  of  UTs  approaches  statistical  significance  (p=.12).  The  latter  finding  suggests  that  the 
practice  in  Trial  1  may  have  limited  the  ability  to  detect  structural  differences  in  the  conduct  of 
unanticipated  tasks  in  Trial  2,  because  the  degree  of  uncertainty  was  reduced  as  a  result  of  teams’ 
conduct  of  the  scenario  with  UTs  in  Trial  1.  The  lack  of  a  training  effect  for  accuracy  scores  is 
potentially  the  result  of  a  ceiling  effect,  given  that  many  of  these  scores  are  above  90%  for  both 
trials. 


Table  6.  Simulation  Performance:  Impact  of  Training  (Order  Effect) 

Trial  1 

Trial  2 

Sig.  Level 

Mission  score 

91.0 

(6.2) 

91.0 

(5.8) 

n.s. 

Strength  score 

86.5 

(6.06) 

90.3 

(3.6) 

n.s. 

7  Primary  Offensive  Mission  Tasks  (predictable)  -Accuracy 

92.4 

(11-5) 

97.0 

(5.1) 

n.s. 

7  Primary  Offensive  Mission  Tasks  (predictable)  -Latency 

1131 

(114) 

982 

(69) 

p=.oi 

4  Complex  Unanticipated  Tasks  (unpredictable)  -Accuracy 

83.0 

(32.5) 

93.9 

(15.7) 

P=.12  (n.s.) 

4  Complex  Unanticipated  Tasks  (unpredictable)  -Latency 

313 

(80) 

273 

(75) 

P=.ll  (n.s.) 

Single-node  Defensive  Tasks  -Accuracy 

85.5 

(8.6) 

94.1 

(4.0) 

P=.03 

Single-node  Defensive  Tasks  -Latency 

201 

(35) 

173 

(29) 

P=.07 

Coordinating-nodes  Defensive  Tasks  -Accuracy 

77.3 

(12.7) 

76.4 

(1L1) 

n.s. 

Coordinating-nodes  Defensive  Tasks  -Latency 

479 

(143) 

279 

(63) 

E<.01 

Note:  Means  and  (std  dev.).  Accuracy  based  on  100%;  Latency  is  number  of  seconds. 

Paired  t-test  comparison  of  means,  1 -tailed  significance  levels.  N=6. 

Table  7  compares  observer  ratings  of  teamwork  processes  across  the  two  trials.  Of  the  ten 
measures  shown,  half  show  Trial  2  as  significantly  higher  than  Trial  1  (pc.  10),  with  two 
additional  measures  showing  marginal  significance.  These  findings  suggest  that  teams  continue 
to  enhance  their  teamwork  processes  with  each  exposure  to  the  mission  simulation.  It  is  likely 
that  these  improvements  in  teamwork  contribute  to  the  improvements  in  performance  latency 
shown  in  Table  6. 


Table  8  presents  the  participants’  ratings  of  the  organizational  and  process  characteristics 
compared  by  trial.  Items  13  and  14  are  most  relevant  to  the  question  posed  at  the  beginning  of 
this  section  and  show  that,  indeed,  the  preparation  for  UTs  was  significantly  higher  in  Trial  2 


than  Trial  1.  This  suggests  that  the  Trial  2  data  cannot  be  taken  as  a  satisfactory  test  of  the 
research  question  regarding  the  link  between  structure  and  performance  of  complex  unpredicted 
tasks.  In  addition,  participant  ratings  give  specific  insights  as  to  which  characteristics  were  most 
influenced  by  the  training  effect  (and  which  were  not). 


Table  7.  Observer  Ratings  of  Processes  Comparin; 

g  Two  Structures 

Trial  1 

Trial  2 

T-test 

significance 

Overall  Teamwork 

4.90 

5.80 

pc.06* 

Members  anticipate  others’  need  for  assistance 

4.75 

5.42 

p=.15  (n.s.) 

Members  adjust  to  redistribute  workload 

4.58 

5.50 

pc. 08  * 

Communication  Effectiveness 

4.67 

5.42 

n.s. 

Proactive  Communication 

4.83 

5.75 

pc. 07* 

Coordination 

4.83 

6.50 

pc. 04* 

Team  Orientation 

5.92 

6.08 

n.s. 

Willingness  to  shift  from  main  mission  tasks  to  UTs 

5.83 

5.98 

n.s. 

Extent  to  which  team  considered  alternatives  in 
processing  UTs 

4.56 

5.20 

pc.  11  (n.s.) 

Teams  level  of  activity  when  UTs  appeared 

4.54 

5.25 

pc. 05* 

Note:  Means  based  on  a  rating  scale  where  1=  “poor”  and  7=  “superior.” 

*  Difference  significant  using  independent  samples  t-test;  1-tailed  significance  (pc. 10).  N=6. 

In  addition  to  the  two  mentioned  above,  the  three  factors  that  increased  significantly  from  Trial  1 
to  Trial  2  were:  the  development  and  use  of  strategies  for  processing  tasks,  situational 
awareness,  and  willingness  to  shift  from  planned  tasks  to  UTs.  It  is  noteworthy  that  participant 
ratings  do  not  show  a  significant  order  effect  on  task  performance  outcomes  for  either  planned 
mission  tasks  or  unanticipated  tasks.  And,  unlike  the  observer-based  findings,  the  differences  for 
coordination  and  communication  are  not  significantly  different.  Overall,  the  findings  reported  in 
Tables  6-8  show  clear  evidence  of  a  training  effect  particularly  in  the  areas  of  teamwork 
strategies,  preparedness  for  UTs,  and  improved  speed  of  task  accomplishment  (i.e.,  latencies). 


Table  8.  Participant  Ratings  of  Performance  Comparing  Trial  1  and  Trial  2 

Trial  1 

Trial  2 

1.  Mission  performance? 

5.50 

5.57 

2.  UT  performance? 

5.55 

5.70 

3.  Flexibility 

5.09 

5.03 

4.  Facilitation  of  coordination 

5.23 

5.40 

5.  Facilitation  of  communication 

5.25 

5.38 

6.  Facilitation  of  development/use  of  efficient  strategies 

5.13 

5.60* 

7.  Balance  of  task  distribution 

4.89 

5.02 

8.  Situational  awareness 

5.19 

5.49* 

9.  Extent  to  which  UTs  interfered  with  execution  of  planned  mission  tasks 

4.43 

4.45 

10.  Willingness  to  shift  from  planned  tasks  to  UTs 

5.70 

5.90* 

11.  To  what  extent  did  team  consider  alternatives  in  processing  UTs? 

4.36 

4.56 

12.  Degree  of  Flag  involvement  in  processing  UTs? 

5.45 

5.25 

13.  How  prepared  for  first  UT? 

4.51 

5.60* 

14.  How  prepared  for  subsequent  UTs? 

5.26 

5.66* 

Note:  Means  are  based  on  a  rating  scale  where  1=  “low”  and  7  =  “high”  value  on  the  characteristic. 

*  paired  t-test  significant  pc.  10  ( 1-tailed).  All  other  t-tests  not  significant. 

N's  ranged  from  29  to  36  given  missing  cases. 

4.  Discussion 


The  primary  purpose  of  this  experiment  was  to  examine  the  relationship  between  structure  (as 
differentiated  by  degree  of  practiced  coordination)  and  performance  of  tasks  that  varied  in  terms 
of  complexity  and  predictability.  Previous  experiments  had  shown  that  a  model-based  structure 
that  was  designed  for  increased  unit-level  autonomy  (A  1-6)  outperformed  a  more  interdependent, 
functional  structure  (AO-6)  on  the  complex,  but  predictable  mission  tasks.  This  finding  was  not 
consistently  supported  by  the  current  experiment.  Only  observer  ratings  showed  the  model- 
based  Al-6  significantly  outperforming  the  traditional,  functional  AO-6  on  mission  tasks.  The 
comparison  of  the  two  structures  on  the  complex  Unanticipated  Tasks  also  shows  only  limited 
differences.  The  prediction  derived  from  organization  theory  that  greater  coordination  capability 
would  facilitate  the  ability  to  respond  to  complex,  unanticipated  events  was  only  supported  by 
significant  differences  in  the  participants’  ratings  favoring  the  more  interdependent  AO-6. 

There  are  several  possible  explanations  for  these  results.  First,  the  structural  changes  made  to 
remove  the  communication  networks  used  in  the  previous  experiment  may  have  given  both 
structures  greater  situational  awareness.  This  improved  common  operational  picture  and  the 
ability  to  communicate  freely  across  the  entire  structure  may  have  overridden  the  differences  in 
structure  based  on  either  task-based  or  functional  asset  allocations.  A  second  possible 
explanation  is  that  there  is  not  sufficient  differentiation  in  the  amount  of  coordination  “practiced” 
by  the  two  structures  to  test  the  role  of  this  capability  in  situations  of  task  uncertainty.  A  third 
factor  evident  is  that  of  a  ceiling  effect,  particularly  for  accuracy  scores.  From  both  data  and 
participant  comments,  the  challenge  of  the  scenario  may  not  have  been  sufficient  to  allow 
performance  differences  between  the  structures  to  emerge.  This  is  particularly  problematic  in 
small  sample  experiments  such  as  this,  with  limited  statistical  power. 

The  final  possible  explanation  for  limited  results  by  structure  is  related  to  the  ceiling  effect,  but 
also  has  theoretical  implications.  This  is  the  role  of  training  on  performance.  As  shown  in  the 
final  section  of  the  results  reported  above,  there  were  many  more  significant  differences  in  terms 
of  both  performance  outcomes  and  teamwork  processes  in  comparing  Trial  1  and  Trial  2  than 
there  were  for  the  comparisons  by  structure.  This  suggests  that  teams  were  continuing  to 
develop  skills  that  enhanced  their  performance.6  The  testable  hypothesis  to  be  derived  from  this 
is  that  the  continuing  experience  of  rehearsal  was  more  powerful  in  determining  performance 
than  the  differential  characteristics  of  the  two  structures.  The  interesting  theoretical  question  is 
to  identify  what  particular  skills  are  most  salient  in  improving  performance.  Previous  research 
such  as  that  by  Weick  and  Roberts  (1993)  on  the  coordination  used  in  the  conduct  of  operations 
on  the  flight  deck  of  aircraft  carriers  can  provide  insights  as  to  potential  critical  dimensions  of 
coordination  capability. 

The  discussion  above  has  clear  implications  for  future  research.  To  further  test  the  relationship 
between  coordination  structures  and  task  performance,  the  following  changes  to  research  method 
are  suggested.  The  ceiling  effect  can  be  addressed  by  increasing  the  challenge  of  the  scenario 
through  increased  task  demand  and  competing  requirements.  The  operationalization  of 


6  It  is  important  to  note  that  the  research  design  was  balanced  in  terms  of  the  order  in  which  teams  were  assigned 
structures  for  their  two  simulation  trials.  Thus,  the  influence  of  training  did  not  differentially  effect  either  structure. 


“coordination  capability”  can  be  made  more  distinctive  by  re-designing  the  two  structures  to 
increase  the  unit-level  autonomy  in  Al-6  and  the  interdependence  in  AO-6.  The  role  of  the 
communication  nets  on  performance  can  be  re-evaluated  by  conducting  the  experiment  reported 
here  using  the  same  structures,  but  with  the  networks  established  as  they  were  in  Experiment  4 
(see  Hocevar  et  ah,  1999). 

A  different  research  implication,  however,  is  suggested  for  evaluating  the  role  team  rehearsal 
plays  on  performance  of  tasks  that  vary  in  terms  of  complexity  and  predictability.  One  question 
could  be  to  test  the  relative  impact  of  rehearsal  of  technical  skills  (e.g.,  asset  use,  information 
technology,  coordination  techniques)  and  interpersonal  skills  (e.g.,  team  familiarity,  cohesion)  on 
performance.  A  possible  research  design  would  be  to  follow  the  same  training  regimen  used  in 
this  experiment  where  teams  learn  both  individual  skills  for  use  of  specific  command  assets  as 
well  as  techniques  of  coordination  (in  terms  of  use  of  software,  requirements  of  scenario,  etc.). 
Then,  for  actual  data  trials,  half  of  the  teams  would  be  “shuffled”  so  that  each  individual  is  still 
assigned  to  the  command  node  for  which  they  trained,  but  they  are  naive  to  their  other  team 
members  in  terms  of  interpersonal  dynamics.  A  comparison  of  teams  of  equivalent  technical 
competence,  but  varied  in  terms  of  interpersonal  familiarity  and  team  cohesion,  could  provide 
important  insights  into  the  training  implications  for  improved  organizational  performance  in 
situations  of  predictability  and  organizational  adaptability  to  situations  of  uncertainty. 

Overall,  the  strongest  finding  from  this  experiment  is  that  training  was  more  important  than 
structure  in  determining  improved  performance.  This  has  significant  potential  implications  for 
practice,  particularly  in  the  current  defense  environment  of  complex  and  interdependent  mission 
areas  requiring  coalition  and  joint  operations.  While  continued  basic  research  on  the  technical 
and  process  aspects  of  coordination  capability  is  needed,  this  research  suggests  that  increased 
focus  on  coordination  training  and  team  building  can  improve  operational  success  in  such 
complex  missions.  The  question  of  the  impact  of  prior  rehearsal  in  team  coordination  on 
performance  can  also  be  examined  in  Tier  2  experiments  such  as  Fleet  Battle  Experiments  or  the 
Global  Wargame. 
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